High precision additive pattern recognition for image and other applications

ABSTRACT

A computer-implemented method includes selecting a kernel and kernel parameters for a first Support Vector Machine (SVM) model, testing the first SVM model on a feature matrix T of n feature vectors of length m to produce false positive (FP) data set and false negative (FN) data set by a computer processor, wherein n and m are integer numbers, automatically removing feature vectors corresponding to the FN data set from the feature matrix T by the computer processor to produce a feature matrix T_best of size (n-size(FN))*m, retraining the first SVM model on the feature matrix T_best to produce a second SVM model, and checking if a ratio (T_best sample number)/(SVM support vector number) is above a threshold for the second SVM model on T_best. If the ratio is above the threshold, SVM predictions is performed using the second SVM model on the feature matrix T_best.

TECHNICAL FIELD

This application relates to improved image recognition methods forautomated imaging and other applications.

BACKGROUND OF THE INVENTION

The wide adoption of smart phones and digital cameras have caused anexplosion in the number of digital images. Digital images can be viewed,shared over the Internet, and posted on mobile applications and socialnetworks using computer devices. Images can also be incorporated intothe designs of photo products such as photographic prints, greetingcards, photo books, photo calendars, photo mug, photo T-shirt, and soon. In electronic forms or on physical products, images are often mixedwith text and other design elements, and laid out in a particularfashion to tell a story.

Digital images often contain significant objects and people's faces.Creating photo blogs or high-quality image products, such as photobooksand greeting cards, naturally requires proper consideration of thoseobjects and people's faces. For example, the most important and relevantpeople such as family members should have their faces prominently shownin image products while strangers' faces minimized. In another example,while pictures of different faces at a same scene can be included in animage-based product, the pictures of a same person at a same sceneshould normally be trimmed to allow only the best one(s) to bepresented. Significant objects can include people's clothing, dailyobjects such as furniture, a birthday cake, and a soccer ball, andnatural or man-made landmarks.

Faces need to be detected and group based on persons' identities beforethey can be properly selected and placed in image products. Mostconventional face detection techniques concentrate on face recognition,assuming that a region of an image containing a single face has alreadybeen detected and extracted and will be provided as an input. Commonface detection methods include knowledge-based methods;feature-invariant approaches, including the identification of facialfeatures, texture and skin color; template matching methods, both fixedand deformable; and appearance based methods. After faces are detected,face images of each individual can be categorized into a groupregardless whether the identity of the individual is known or not. Forexample, if two individuals Person A and Person B are detected in tenimages. Each of the images can be categorized or tagged one of the fourtypes: A only; B only, A and B; or neither A nor B. Algorithmically, thetagging of face images require training based one face images of knownpersons (or face models), for example, the face images of family membersor friends of a user who uploaded the images.

To save users' time, technologies have been developed to automate thecreation of photo-product designs. These automatic methods are facingincreased challenges as people take more digital photos. A singleaverage vacation trip nowadays can easily produce thousands of photos.Automatic sorting, analyzing, grouping, and laying out such a greatnumber of photos in the correct and meaningful manner are an immensetask.

There is a need for more accurately recognizing and grouping face imagesand other objects and incorporating them into photo-product designs andother imaging applications.

SUMMARY OF THE INVENTION

Various machine learning methods have been used to recognize andcategorize faces or objects in images. One commonly used method issupport vector machine (SVM). A common problem in SVM and other machinelearning methods is that recognition accuracy is low when the sample setis small. The cause for this problem is that there are typically a largenumber of support vectors; a small sample set leads to data overfitting.Such sample bias results in recognition errors in the form of falsepositives and false negatives.

The present application discloses novel methods that increase the ratioof sample number to support vector number in an SVM model, which canreduce false positives and increases recognition accuracies.

In a general aspect, the present invention relates to acomputer-implemented method that includes selecting a kernel and kernelparameters for a first Support Vector Machine (SVM) model, testing thefirst SVM model on a feature matrix T to produce false positive (FP)data set and false negative (FN) data set by a computer processor,wherein the feature matrix T includes n feature vectors of length m,wherein n and m are integer numbers. The method includes automaticallyremoving feature vectors corresponding to the FN data set from thefeature matrix T by the computer processor to produce a feature matrixT_best of size (n-size(FN))*m, retraining the first SVM model on thefeature matrix T_best to produce a second SVM model, checking if a ratio(T_best sample number)/(SVM support vector number) is above a thresholdfor the second SVM model on T_best, and if the ratio is above thethreshold, performing SVM predictions using the second SVM model on thefeature matrix T_best.

Implementations of the system may include one or more of the following.If the ratio (T_best sample number)/(SVM model support vector number) isnot above the threshold for the SVM model on T_best, repeating the stepsof selecting, testing, automatically removing, retraining, and checkingto find a third SVM model having (T_best sample number)/(SVM modelsupport vector number) ratio that exceeds the threshold. Thecomputer-implemented method can further include performing SVMpredictions using the third SVM model on the feature matrix T_best. Thecomputer-implemented method can further include: if a SVM model having(T_best sample number)/(SVM model support vector number) ratio thatexceeds the threshold is not found, selecting a fourth SVM model thatyields a highest (T_best sample number)/(SVM model support vectornumber) ratio in the step of repeating the steps of selecting, testing,automatically removing, retraining, and checking; and performing SVMpredictions using the third SVM model on the feature matrix T_best. Thethreshold for the (T_best sample number)/(SVM model support vectornumber) ratio can be ten or higher. The computer-implemented method canfurther include applying Grid Search on the feature matrix T to findoptimal kernel parameters before the step of testing the first SVM modelon a feature matrix T. The feature vectors can be based on faces orobjects in images; the computer-implemented method can further includeclassifying the objects and the faces in the images by performing SVMpredictions on the feature matrix T_best. The computer-implementedmethod can further include automatically creating designs for photoproducts based on the objects and faces classified by performing SVMpredictions on the feature matrix T_best.

The computer-implemented method can further include selecting a featurevector v from the FN data set to add back to the feature matrix T_bestto produce a feature matrix T_v, retraining the second SVM model on thefeature matrix T_v to produce a fifth SVM model, performing SVMpredictions on the feature matrix T_v using the fifth SVM model,calculating a benefit function to produce a benefit function value,wherein the benefit function is dependent of differences between truepositives, true negatives, and numbers of support vectors generated onT_best and T_v respectively by the second SVM model and the fifth SVMmodel, adding the feature vector v to T_best if the benefit functionvalue meets a predetermined criterion; and performing SVM predictionusing the feature matrix T_best. The computer-implemented method canfurther include repeating the steps of selecting a feature vector v,training the second SVM model, performing SVM predictions, calculating abenefit function, and adding the feature vector v to T_best by selectingand adding a different feature vector from the FN data set to featurematrix T_best, wherein SVM predictions are performed using the featurematrix T_best that gives a highest best function value. All the featurevectors corresponding to the FN data set can be evaluated by calculatinga corresponding benefic function, where are respectively determined tobe added to the T_best or not depending on a value of the correspondingbenefic function. The feature vector v is not added to the featurematrix T_best if the value of the benefit function does not meet thepredetermined criterion. The feature vectors can be based on faces orobjects in images, the computer-implemented method can further includeclassifying the objects and the faces in the images by performing SVMpredictions on the feature matrix T_best.

These and other aspects, their implementations and other features aredescribed in detail in the drawings, the description and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a network-based system for producingpersonalized image products and photo blogs and shares compatible withthe present invention.

FIG. 2 is a flow diagram illustrating a high-precision SVM method withincreased recognition accuracy in accordance with the present invention.

FIG. 3 is a flow diagram illustrating an additive high-precision SVMmethod for reducing false positives and increasing recognition accuracyin accordance with the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Referring to FIG. 1, a network-based imaging service system 10 operatedby an image service provider such as Shutterfly, Inc. enables users 70,71 to store, organize, and share images via a wired network or awireless network 51. The network-based imaging service system 10 alsoallows users 70, 71 to design image products.

The network-based imaging service system 10 includes a data center 30,one or more product fulfillment centers 40, 41, and a computer network80 that facilitates the communications between the data center 30 andthe product fulfillment centers 40, 41.The data center 30 includes oneor more servers 32 for communicating with the users 70, 71, a datastorage 34 for storing user data, image and design data, and productinformation, and computer processor(s) 36 for rendering images andproduct designs, organizing images, and processing orders. The user datacan include account information, discount information, and orderinformation associated with the user. A website can be powered by theservers 32 and can be accessed by the user 70 using a computer device 60via the Internet 50, or by the user 71 using a wireless device 61 viathe wireless network 51. The servers 32 can also support a mobileapplication to be downloaded onto wireless devices 61.

The network-based imaging service system 10 can provide products thatrequire user participations in designs and personalization. Examples ofthese products include the personalized image products that incorporatephotos provided by the users, the image service provider, or othersources. In the present disclosure, the term “personalized” refers toinformation that is specific to the recipient, the user, the giftproduct, and the occasion, which can include personalized content,personalized text messages, personalized images, and personalizeddesigns that can be incorporated in the image products. The content ofpersonalization can be provided by a user or selected by the user from alibrary of content provided by the service provider. The term“personalized information” can also be referred to as “individualizedinformation” or “customized information”.

Personalized image products can include users' photos, personalizedtext, personalized designs, and content licensed from a third party.Examples of personalized image products may include photobooks,personalized greeting cards, photo stationeries, photo or image prints,photo posters, photo banners, photo playing cards, photo T-shirts, photomugs, photo aprons, photo magnets, photo mouse pads, a photo phone case,a case for a tablet computer, photo key-chains, photo collectors, photocoasters, photo banners, or other types of photo gift or novelty item.The term photobook generally refers to as bound multi-page product thatincludes at least one image on a book page. Photobooks can include imagealbums, scrapbooks, bound photo calendars, or photo snap books, etc. Animage product can include a single page or multiple pages. Each page caninclude one or more images, text, and design elements. Some of theimages may be laid out in an image collage.

The user 70 or his/her family may own multiple cameras 62, 63. The user70 transfers images from cameras 62, 63 to the computer device 60. Theuser 70 can edit, organize the images on the computer device 60, whichcan include a personal computer, a laptop, tablet computer, a mobilephone, etc. The cameras 62, 63 include a digital camera, a camera phone,a video camera, as well as image capture devices integrated in orconnected with in a computer device, such as the built-in cameras inlaptop computers or computer monitors. The user 70 can also printpictures using a printer 65 and make image products based on the imagesfrom the cameras 62, 63.

Images can be uploaded from the computer device 60 and the wirelessdevice 61 to the server 32 to allow the user 70 to organize and renderimages at the website, share the images with others, and design or orderimage product using the images from the cameras 62, 63. If users 70, 71are members of a family or associated in a group (e.g. a soccer team),the images from the cameras 62, 63 and the mobile device 61 can begrouped together to be incorporated into an image product such as aphotobook, or used in a blog page for an event such as a soccer game.

Products can be ordered by the users 70, 71 based on the image productdesigns. The physical products can be manufactured by the printing andfinishing facilities 40 and 41 and received by recipients at locations180, 185. The recipients can also receive digital versions of theimage-product designs over the Internet 50 and/or a wireless network 51.For example, the recipient can receive, on her mobile phone, anelectronic version of a greeting card signed by handwritten signaturesfrom her family members.

For the fulfillments of personalized image products, the productfulfillment center 40 can include one or more printers 45 for printingimages, finishing equipment 46 for operations such as cutting, folding,binding the printed image sheets, and shipping stations 48 for verifyingthe orders and shipping the orders to recipients 180 and 185. Examplesof the printers 45 include can be digital photographic printers, offsetdigital printers, digital printing presses, and inkjet printers. Thefinishing equipment 46 can perform operations for finishing a completeimage-based product other than printing, for example, cutting, folding,adding a cover to photo book, punching, stapling, gluing, binding, andenvelope printing and sealing. The shipping stations 48 may performtasks such as packaging, labeling, package weighing, and postagemetering.

The creation of personalized image products can take considerable amountof time and effort. To save users' time, technologies have beendeveloped by Shutterfly, Inc. to automate the creation of photo-productdesigns. Given the thousands of photos that people now often take justfrom one occasion, automatic sorting, analyzing, grouping, curating, andlaying out a large number of photos have become increasinglychallenging. Accurately recognizing faces and objects in photos areessential to the technologies for automated photo-product designs. Tothis end, an improved machine learning method based on SVM is describedbelow, which has helped the recognition accuracies in classifying andrecognizing objects or faces, especially when only a small number ofsample objects or faces are available in those images.

A common problem in SVM and other machine learning methods is thatrecognition accuracy is low when the sample set is small. The cause forthis problem is that there are typically a large number of supportvectors; a small sample set can lead to data overfitting, resultingfalse positives and false negatives. Precision and Recall are twometrics for measuring classifier output quality. Higher Precision can beachieved by reducing false positives, while higher Recall is obtained byreducing false negatives.

In the presently disclosed methods, the over fitting problem is reducedand recognition accuracy improved by tackling one of rather than bothhigher Precision and higher Recall goals. Since it is found that FalsePositives are a main reason for recognition inaccuracies, the disclosednovel SVM methods have been developed by focusing on achieving higherPrecision in classifying face and other objects.

A SVM model includes a kernel function, kernel parameters used fortransform predicted data, and a set of support vectors which are used toseparate the transformed data into True and false predictions.

In some embodiments, FIG. 2 shows a first sub-flow of the disclosedmethods, in which the kernel and kernel parameters are optimized in theSVM model by removing feature vectors corresponding to False Negativesfrom the feature vector matrix and focusing on the reduction of FalsePositive (i.e. High Precision).

Referring to FIGS. 1 and 2, a computer processor selects kernelparameters for Support Vector Machine (SVM) using a feature matrix Tcomprising n feature vectors each having a length m (step 200). Inpattern recognition and machine learning, a feature vector is ann-dimensional vector of numerical features that represent some objects(e.g. a face image or other objects as described above). Representinghuman faces or objects by numerical feature vectors can facilitateprocessing and statistical analysis of the human faces or the objects.The vector space associated with these feature vectors is often calledthe feature space.

A feature matrix comprising all feature vectors represents the entiredata set used for testing, training, and validation of SVM models. Theentire data set is divided into test set, and train set. At every pointon the grid for grid search, a cross validation set is randomly sampledfrom the train set. The SVM model is trained on train set or the crossvalidation set. Precision/recall are evaluated by comparing SVMprediction on the cross validation set with the Ground Truth labels onthat set. The SVM model with the best precision or best recall is beingchosen from the grid search.

The selection of kernel parameters for SVM is conducted on a test dataset. Kernel methods require only a user-specified kernel, i.e., asimilarity function over pairs of data points in raw representation.Kernel methods employ kernel functions to operate in a high-dimensional,implicit feature space without ever computing the coordinates of thedata in that space, but rather by simply computing the inner productsbetween the images of all pairs of data in the feature space. Thisoperation is often computationally cheaper than the explicit computationof the coordinates. The computer processor can be implemented by thecomputer processor 36, the computer device 60, or the mobile device 61.

In the disclosed high-precision SVM method, the computer processorapplies Grid Search on the feature matrix T to find optimal kernelparameters (step 210). A set of pre-prepared “Ground Truth” labels forall feature vectors. The computer processor compares the Ground-Truthlabels to the SVM prediction vector to find TP, TN, FP, FN data sets.The SVM model based on the kernel and the optimized kernel parameters isthen tested on the entire data set, the feature matrix T, to producefalse positive (FP) and false negative (FN) data sets (step 220).

In the disclosed SVM method, in order to achieve high precision, thefeature vectors corresponding to FP data set are kept in the featurematrix so FP can be minimized in the SVM. On the other hand, thosefeature vectors corresponding to FN data set are tested in vs. out ofthe data set to produce the highest accuracy (high precision). Thecomputer processor then automatically removes those feature vectorscorresponding to the FN data set in the feature matrix T to produce anew feature matrix T_best of size (n-size(FN)) * m (step 230), wherein nand m are integer numbers.

The computer processor retrains the SVM model on T_best using the kerneland the kernel parameters optimized in step 210 to produce a new SVMmodel (step 240).

The new SVM model is then evaluated on T_best. The computer processorchecks if the ratio (T_best sample number)/(SVM support vector number)is above a threshold for the new SVM model on T_best by the computerprocessor (step 250). For example, the threshold for the ratio can be 10or higher, meaning that it is desirable for the sample number to be atleast ten times of the support vector number in the SVM model to avoidthe overfitting problem.

If the ratio is not above the threshold, steps 200-250 are repeated tofind a SVM model (with new kernel parameters and sometimes a new kernel)having (T_best sample number)/(SVM support vector number) ratio thatexceeds the threshold (step 260). This iteration may involve trying andselecting different kernel parameters, and/or using a larger data set.

If such a SVM model having the new ratio above the threshold can befound in steps 260, that SVM model is selected (step 270). Otherwise, ifsuch a ratio cannot be found, the computer processor selects the SVMmodel having kernel and the kernel parameters that have yielded thehighest ratio in step 260 (step 270).

The computer processor can then perform SVM predictions using theresulting SVM model on the entire data set of the feature matrix T_best(step 280). The resulting SVM can be the SVM found in steps 250 and 260that has the ratio exceeding the threshold, or the SVM model that hasyielded the highest ratio in step 260. In some embodiments, the featurevectors are based on faces or objects in sample images. The objects andthe faces in the sample images are more accurately classified by SVMpredictions using the resulting SVM model on T_best.

In some embodiments, FIG. 3 shows a second sub-flow of the disclosedmethods, in which the support vector set is optimized in the SVM model.The first sub-flow of the disclosed high-precision SVM method describedabove in relation to in FIG. 2 can be improved further in accuracies byoptimizing the placements of the feature vectors corresponding to the FNdata set in vs. out of the feature matrix T_best. The accuracies of theSVM model is improved, while the complexity of the SVM model (number ofsupport vectors) is reduced.

Referring to FIGS. 1-3, a feature vector v from the FN data set isselected and added back to feature matrix T_best to produce a newfeature matrix T_v (step 300). The feature matrix T_v is a temporaryfeature matrix for evaluating the feature vector v.

The SVM model based on the kernel and the optimized kernel parametersadopted in step 270 is retrained on the feature matrix T_v to produce anew SVM model (step 310). The computer processor then performs SVMpredictions using the new SVM model on the feature matrix T_v (step320).

A benefit function value is then calculated (step 330). The benefitfunction is dependent on the differences between true positives, truenegatives, and the number of support vectors respectively generated bythe old SVM model and the new SVM model respectively generated on T_bestand T_v (step 330).

For instance, an example of such benefit function is as follows:

C(v) = [size(FP_best) − size(FP_v)] * alpha +   [size(FN_best) − size(FN_v)] * beta + [nSV_best − nSV_v] * gamma,

where alpha, beta and gamma are positive global parameters, used tocontrol the ratio of sample number to support vector number. Decreasesin False sets (Positive or Negative) and a decrease in model complexity(nSV, the number of support vectors) from T_best to T_v are beneficial,and will result in an increased value of the benefit function C(v).

An example for the set of (alpha, beta, gamma) values are (2, 1, 5). Inthis example, alpha>beta means that FP is more significant than FN, andan increase in FP (i.e. a decrease in TP) should be penalized more,which is intended for the present high-precision SVM method. A largergamma makes sure that support vectors are not added without a strongreason.

The feature vector v is added to T_best if the value of the benefitfunction meets a predetermined criterion (step 340). For example, thethreshold value can be zero for the benefit function. If C(v)>0, thenthe decrease in false sets is worthwhile and the model complexity hasnot jumped. If so, the new feature vector v is added to T_best (step340), and the True sets, False sets, and the feature vectors are updatedin the feature matrix:

FN_best=FN_v

TN_best=TN_v

FP_best=FP_v

TP_best=TP_v

nSV_best=nSV_v.

In other words, T_best is the current best feature vector matrix thatincludes all feature vector v that have been determined to be beneficialusing the method describe above.

If the benefit function value does not meet the predetermined criterion,the feature vector v is not added to the feature matrix T_best (step350). In the example above, if C(v)<=0, the True sets, False sets, andthe feature vectors in the feature matrix are not updated.

Then steps 300-350 is repeated by selecting and adding a differentfeature vector from the FN data set to the feature matrix T_best (step360). All the feature vectors corresponding to the FN data set that havebeen removed in step 230 can be evaluated in an iteration by repeatingsteps 300-360.

After all feature vectors corresponding to the FN data set have beenevaluated as discussed above, the computer processor performs SVMprediction using the feature matrix T_best that gives the highest bestfunction value (step 370). If no feature vector v has been added toT_best, T_best is still used for performing SVM predictions as in steps270-280.

In some embodiments, the feature vectors are based on faces or objectsin sample images. The objects and the faces in the sample images aremore accurately classified by SVM predictions using the resulting SVMmodel on T_best.

After the faces and objects in the sample images are classified by theSVM model obtained in step 280 and step 370, the images can be properlyselected to be incorporated in image-product designs. The curation andlayout of the image-product designs can be automatically created by thecomputer processor 36, the computer device 60, or the mobile device 61,then presented to a user 70 or 71 (FIG. 1), which allows the imageproduct to be ordered and manufactured by the printing and finishingfacilities 40 and 41 (FIG. 1). The image product creation can alsoinclude partial user input or selections on styles, themes, formats, orsizes of an image product, or text to be incorporated into an imageproduct.

The accurate recognition and grouping of faces and objects disclosedherein can significantly reduce time to create the product designs, andimprove the relevance and appeal of an image product. For example, themost important people can be determined and to be emphasized in an imageproduct. Redundant person's face images can be filtered out and selectedbefore incorporated into an image product. Irrelevant persons can beminimized or avoided in the image product.

The disclosed methods can include one or more of the followingadvantages. The disclosed methods can drastically increase theaccuracies in classifying objects and faces in images, especially insmall sample set. The disclosed methods can be implementedautomatically, and can produce image-based product designs with higherquality, better appeal, and more relevance to users.

It should be noted that the presently disclosed methods are not limitedto imaging applications and image-product designs. Examples of othersuitable applications include information classification and prediction,text recognition and classification, voice recognition, biometricsrecognition, classification of gene expression data and other biomedicaldata, etc.

What is claimed is:
 1. A computer-implemented method, comprising:selecting a kernel and kernel parameters for a first Support VectorMachine (SVM) model; testing the first SVM model on a feature matrix Tto produce false positive (FP) data set and false negative (FN) data setby a computer processor, wherein the feature matrix T includes n featurevectors of length m, wherein n and m are integer numbers; automaticallyremoving feature vectors corresponding to the FN data set from thefeature matrix T by the computer processor to produce a feature matrixT_best of size (n-size(FN))*m; retraining the first SVM model on thefeature matrix T_best to produce a second SVM model; checking if a ratio(T_best sample number)/(SVM support vector number) is above a thresholdfor the second SVM model on T_best; and if the ratio is above thethreshold, performing SVM predictions using the second SVM model on thefeature matrix T_best.
 2. The computer-implemented method of claim 1,comprising: if the ratio (T_best sample number)/(SVM model supportvector number) is not above the threshold for the SVM model on T_best,repeating the steps of selecting, testing, automatically removing,retraining, and checking to find a third SVM model having (T_best samplenumber)/(SVM model support vector number) ratio that exceeds thethreshold
 3. The computer-implemented method of claim 2, comprising:performing SVM predictions using the third SVM model on the featurematrix T_best.
 4. The computer-implemented method of claim 2,comprising: if a SVM model having (T_best sample number)/(SVM modelsupport vector number) ratio that exceeds the threshold is not found,selecting a fourth SVM model that yields a highest (T_best samplenumber)/(SVM model support vector number) ratio in the step of repeatingthe steps of selecting, testing, automatically removing, retraining, andchecking; and performing SVM predictions using the third SVM model onthe feature matrix T_best.
 6. The computer-implemented method of claim1, wherein the threshold for the (T_best sample number)/(SVM modelsupport vector number) ratio is ten or higher.
 7. Thecomputer-implemented method of claim 1, comprising: applying Grid Searchon the feature matrix T to find optimal kernel parameters before thestep of testing the first SVM model on a feature matrix T.
 8. Thecomputer-implemented method of claim 1, wherein the feature vectors arebased on faces or objects in images, the computer-implemented methodfurther comprising: classifying the objects and the faces in the imagesby performing SVM predictions on the feature matrix T_best.
 9. Thecomputer-implemented method of claim 8, further comprising:automatically creating designs for photo products based on the objectsand faces classified by performing SVM predictions on the feature matrixT_best.
 10. The computer-implemented method of claim 1, furthercomprising: selecting a feature vector v from the FN data set to addback to the feature matrix T_best to produce a feature matrix T_v;retraining the second SVM model on the feature matrix T_v to produce afifth SVM model; performing SVM predictions on the feature matrix T_vusing the fifth SVM model; calculating a benefit function to produce abenefit function value, wherein the benefit function is dependent ofdifferences between true positives, true negatives, and numbers ofsupport vectors generated on T_best and T_v respectively by the secondSVM model and the fifth SVM model; adding the feature vector v to T_bestif the benefit function value meets a predetermined criterion; andperforming SVM prediction using the feature matrix T_best.
 11. Thecomputer-implemented method of claim 10, further comprising: repeatingthe steps of selecting a feature vector v, training the second SVMmodel, performing SVM predictions, calculating a benefit function, andadding the feature vector v to T_best by selecting and adding adifferent feature vector from the FN data set to feature matrix T_best,wherein SVM predictions are performed using the feature matrix T_bestthat gives a highest best function value.
 12. The computer-implementedmethod of claim 11, wherein all the feature vectors corresponding to theFN data set are evaluated by calculating a corresponding beneficfunction and determined to be added to the T_best or not depending on avalue of the corresponding benefic function.
 13. Thecomputer-implemented method of claim 10, wherein the feature vector v isnot added to the feature matrix T_best if the value of the benefitfunction does not meet the predetermined criterion.
 14. Thecomputer-implemented method of claim 10, wherein the feature vectors arebased on faces or objects in images, the computer-implemented methodfurther comprising: classifying the objects and the faces in the imagesby performing SVM predictions on the feature matrix T_best.